1. Field of the Invention
The present invention relates to the field of electronic video and live performance entertainment. More specifically, the present invention relates to entertainment and educational systems whereby participants interact with an electronic or computerized environment in addition to live and/or pre-recorded performers. Even more specifically, the present invention relates to immersive virtual reality computer systems in which participants interact with a virtual reality environment and live performers using a variety of immersion and input devices such as a head mounted display and a handheld keypad.
2. Description of the Related Art
Virtual reality entertainment systems utilizing head mounted displays or other immersive devices or methods, whereby a participant views and interacts with a software-generated virtual reality environment have been used for training and entertainment purposes for many years. An immersive virtual reality environment as used and described herein refers to a computer generated graphical environment wherein a participant is xe2x80x9cimmersedxe2x80x9d within the environment so as to provide to the user a sensation of being physically located within the graphical environment, although the participant is only electronically present with other objects within the environment. Therefore, an immersive virtual reality environment creates an illusion to the participant that he or she is in an artificially created environment through the use of three-dimensional (3D) graphics and computer software which imitates the relationship between the participant and the surrounding environment.
Traditionally, participants are represented in the software environment by projections of figures referred to in the art as avatars. Participants control their avatars through a combination of input mechanisms including, but not limited to, hand held input devices and data generated from electronic and electromagnetic tracking devices which monitor bodily movement. Objects within the virtual reality environment which are not controlled by the motion of the participants are generally controlled by a computer software program and can also respond to the input of the participants.
In related art virtual reality environments, figures or objects not controlled by participants move autonomously only in accordance with the software program and respond to the input devices of each participant (Carmein U.S. Pat. No. 5,490,784, Latypov U.S. Pat. No. 5,846,134). Passive and active objects such as trees and animals move in a predetermined manner within the virtual reality environment. Furthermore, the participant has the opportunity to interact with such objects in the virtual reality environment, however, the objects are limited to computer generated images.
Generally, the object or focus of virtual reality environments is traditionally on the participants"" interaction with objects generated entirely by the computer graphics program that do not represent, or are not substantially controlled by, a human participant in the simulation. Also traditionally, groups of participants have been entertained by graphically enhanced live or pre-recorded performers on stage or through television, cinema or other non-immersion mediums without the enhancements of immersion devices or methods, and furthermore without immersion devices or methods in combination with voice communication from the participants to the performer and amongst and between participants. (Trumbull et al. U.S. Pat. No. 5,795,228).
Other known entertainment systems, such as that which is disclosed in Freeman et al. U.S. Pat. No. 5,682,196, include the capacity for two-way communications between a computer-controlled host and a participant. These systems, however, do not possess the capability for voice communication amongst and between participants, or between participants and a live host. The system in Trumbull allows for one-way audio communication from the live performer to the participants but does not provide for audio communication from the participants to the live performer. Nor does the system of Trumbull allow for audio communication amongst and between the participants, furthermore enhanced by an immersive virtual reality environment.
Several known art virtual reality systems have developed an elementary tactile interface. For example, Shaw et al U.S. Pat. No. 5,737,505 discloses an interface resembling an upper torso suit of armor which detects limb and body position. The device can provide force feedback or impart motion to a participant who is engaging in both active and passive communications with the virtual reality environment. The virtual reality system of Shaw, however, does not provide communication amongst and between participants, nor does the system include live and/or pre-recorded performers.
The known art entertainment systems, therefore, lack certain advantageous features. In traditional systems, the participant may be interacting with a host and the environment using some input device. The host, however, is seldom live and is instead pre-recorded or computer-controlled, therefore reducing the spontaneity of any interaction with the participant. The host in Trumbull can be a live performer, but the system itself is not within a virtual reality environment and therefore lacks the immersive effects which are imparted upon the participants for a more exciting experience. Further, there is no voice communication from the participants to the performer, nor is there voice communication amongst and between participants in Trumbull, thereby further limiting the entertainment or educational experience.
Accordingly, there appears a need in the art for a virtual reality based entertainment and/or educational system which provides three-way immersive interactive communication amongst and between: 1). participants; 2). an immersive environment; and 3). live and/or pre-recorded performers. Further, there appears a need for a system which includes provisions for individual and/or mixed video, audio, and graphical inputs and outputs to enhance the overall entertainment and/or educational experience. Additionally, there is a need for the content and outcome of the experience to be controlled, at least in part, by the inputs of the participants in response to scripted and/or impromptu communications from the performers and/or the virtual reality environment. Finally, there appears to be a need for a system which is robust and modular in design such that a plurality of participants, performers, and virtual reality environment inputs and outputs can be easily retrofitted depending on the desired application and to accommodate the rapid changes of computer technology. The synergistic combination of these elements can provide a hitherto-unrealized level of entertainment and/or educational excitement and quality for an audience of participants.
According to the present Virtual Reality Performance Theater, (VRPT), the inventors change the focus of the software program and entertainment or educational performance from the interaction between individual participants and computer generated graphical objects within a virtual reality environment to a three-way interactive communication between participants, live and/or pre-recorded performers, and an immersive virtual reality environment. The inclusion of three-way communication amongst and between participants, live and/or pre-recorded performers, and an immersive virtual reality environment results in a synergistic effect which creates an unparalleled entertainment and/or educational experience.
The primary object of the present invention, therefore, is to provide an interactive virtual reality performance theater wherein there exists three-way communication amongst and between participants(), live and/or pre-recorded performer(s), and the virtual reality environment. The three-way communication includes, but is not limited to, audio, video, and graphical input and output data. Importantly, the interaction of the participants and performers with each other and the virtual reality environment determines, to some extent, both the content and the ultimate outcome of the performance.
Accordingly, a further object of the present invention is to create an interactive virtual reality environment in which a plurality of participants enjoy immersion in a software-generated simulation combined with live and/or pre-recorded performances.
A further object of the present invention is to allow the participants to exert control over the content and outcome of the performance.
Another object of the present invention is to provide for a novel immersive performance in which each participant and the plurality of performers are linked together for voice communication with a voice intercom system. The system should allow participants to establish a constant dialogue, enhancing interactivity by allowing them to share opinions and work together in an entertainment game or educational setting.
Yet another object of the present invention is to provide a virtual reality environment which is easily customizable to the requirements of a specific application and which can be easily upgraded to comport with the rapid pace of computer technology.
In brief summary, the present invention overcomes and substantially alleviates the deficiencies in the prior art by providing an interactive virtual reality performance platform and environment in which participants can interact with each other, with live and/or pre-recorded performers, and with an immersive virtual reality environment for a highly enjoyable entertainment and/or educational experience. Importantly, participants can exert limited control over the content and outcome of the performance through input devices and voice communication.
The system generally utilizes input and output devices for each of the participants and the performers, and further includes control computers having system data and programs to carry out specific processing tasks. The system further includes a variety of audio and video components which mix and switch audio and video data for presentation to participants in the audience. Through the innovative mixing and switching techniques, the system is able to present to each of the participants a graphical virtual reality environment wherein live video of the performers is superimposed within the graphical environment. The participants, therefore, view and interact with a novel display of both graphical data and live video data for an exciting entertainment and/or educational experience.
In support of the video mixing and switching, the virtual reality display is accomplished through chromakey and other electronically-generated costuming wherein the video and/or audio inputs from the performer(s) is mixed with the graphical environment in which the participant is immersed, using, for example, a head mounted display (HMD). The participants, therefore, experience an immersive graphical environment in which the performer(s) can also be seen and/or heard.
The participants exert control over the content and outcome of the performance through the use of hand-held input devices such as a keypad or cyberglove, while viewing images or electronic facsimiles of the performers/performers, software simulations, and other video, audio, and graphical output of the virtual reality environment. The performer(s), by following a script and/or impromptu actions, query the audience for responses based upon, for example, a specific question asked or their opinions as to a particular subject matter. Participant responses can then dictate the further content and eventually the outcome of the performance, such as a participant being declared the xe2x80x9cwinnerxe2x80x9d based on points earned in responding to the performer(s) and/or system queries.
The system further includes additional audio and motion devices, including but not limited to, speakers and motion-equipped seats for each of the participants to enhance the immersive experience. The immersive experience of the participants is supplemented by aural and physical sensations of the motion-equipped seats, referred to hereinafter as xe2x80x9crumble seats,xe2x80x9d in which the individual participants sit throughout the performance. The seat contains speakers for background sound effects, and seat movement effects are generated by software in response to the participants"" voice communication, decisions, and physical movement.
In addition, a network is included to connect computers for data transmission and to facilitate remote interconnectivity and participation. With implementation of the network, participants and performers can be physically present at geographically remote sites while experiencing the same virtual reality environment and performances.
Accordingly, the inventors add immersive virtual reality technology and electronic interaction to live and/or pre-recorded performances by performers. By placing the participants in a virtual reality environment, the participants and performers can interact more directly while sensing a physical proximity to each other and the immersive virtual reality environment.
An immersive environment, as more fully described herein below, is defined as one in which a greater than 25 degree diagonal field of view is provided no more than 10 feet from the viewer. To achieve an immersive environment, the VRPT can utilize a head mounted display (HMD) device that creates the sensation of immersion in a 3D graphical environment. In addition, specific proximity to a non-head mounted display, such as a monitor at eye level, can create a similar immersive environment. Additional immersion devices and methods are more fully described herein below.
As a result, the entire system can be classified as a blend or meld of virtual reality, computer graphics, and teleconferencing technology. The performers need not be at the local site or sites where the participants are located. Rather, the performers may be located as remotely as communication channels allow and themselves may deliver their performance from different sites to a plurality of sites via the Internet, satellite transfer, or other channels.
One benefit of the present system is that a human participant can interact with other participants and the performers much more spontaneously than in previous forms of interaction. Participants can also have a far more xe2x80x9cpersonal and uniquexe2x80x9d experience through the use of immersion devices and interactive communications technologies.
Another benefit of the present system is that the virtual reality performance theater can now be used as a medium for person-to-person visual projection, as well as entertainment and business applications such as stand-up comedy, dramatic presentations, teaching and lecturing, and other activities related to communication of ideas and concepts to an audience, all within an immersive simulation medium.
Further, the capacity to simultaneously include multiple performers adds to the performance a new dimension. Additional performers are added with audio only, voice only, or both audio and video together. A single performer, although effective, can only engage in monologue and such interaction with the participating audience as the system will permit. By comparison, a plurality of performers in the electronic simulation medium allows this medium to be used for live and recorded dialogue between performers and participants so that true drama may be presented.
Other objects and features will be apparent to those skilled in the art and are pointed out with more particularity herein below.